MDP Optimal Control under Temporal Logic Constraints

In this paper, we develop a method to automatically generate a control policy for a dynamical system modeled as a Markov Decision Process (MDP). The control specification is given as a Linear Temporal Logic (LTL) formula over a set of propositions defined on the states of the MDP. We synthesize a control policy such that the MDP satisfies the given specification almost surely, if such a policy exists. In addition, we designate an "optimizing proposition" to be repeatedly satisfied, and we formulate a novel optimization criterion in terms of minimizing the expected cost in between satisfactions of this proposition. We propose a sufficient condition for a policy to be optimal, and develop a dynamic programming algorithm that synthesizes a policy that is optimal under some conditions, and sub-optimal otherwise. This problem is motivated by robotic applications requiring persistent tasks, such as environmental monitoring or data gathering, to be performed.Comment: Technical report accompanying the CDC2011 submissio

LTL Control in Uncertain Environments with Probabilistic Satisfaction Guarantees

We present a method to generate a robot control strategy that maximizes the probability to accomplish a task. The task is given as a Linear Temporal Logic (LTL) formula over a set of properties that can be satisfied at the regions of a partitioned environment. We assume that the probabilities with which the properties are satisfied at the regions are known, and the robot can determine the truth value of a proposition only at the current region. Motivated by several results on partitioned-based abstractions, we assume that the motion is performed on a graph. To account for noisy sensors and actuators, we assume that a control action enables several transitions with known probabilities. We show that this problem can be reduced to the problem of generating a control policy for a Markov Decision Process (MDP) such that the probability of satisfying an LTL formula over its states is maximized. We provide a complete solution for the latter problem that builds on existing results from probabilistic model checking. We include an illustrative case study.Comment: Technical Report accompanying IFAC 201

Probabilistically safe vehicle control in a hostile environment

In this paper we present an approach to control a vehicle in a hostile environment with static obstacles and moving adversaries. The vehicle is required to satisfy a mission objective expressed as a temporal logic specification over a set of properties satisfied at regions of a partitioned environment. We model the movements of adversaries in between regions of the environment as Poisson processes. Furthermore, we assume that the time it takes for the vehicle to traverse in between two facets of each region is exponentially distributed, and we obtain the rate of this exponential distribution from a simulator of the environment. We capture the motion of the vehicle and the vehicle updates of adversaries distributions as a Markov Decision Process. Using tools in Probabilistic Computational Tree Logic, we find a control strategy for the vehicle that maximizes the probability of accomplishing the mission objective. We demonstrate our approach with illustrative case studies

Optimality and robustness in multi-robot path planning with temporal logic constraints

In this paper we present a method for automatically generating optimal robot paths satisfying high-level mission specifications. The motion of the robot in the environment is modeled as a weighted transition system. The mission is specified by an arbitrary linear temporal-logic (LTL) formula over propositions satisfied at the regions of a partitioned environment. The mission specification contains an optimizing proposition, which must be repeatedly satisfied. The cost function that we seek to minimize is the maximum time between satisfying instances of the optimizing proposition. For every environment model, and for every formula, our method computes a robot path that minimizes the cost function. The problem is motivated by applications in robotic monitoring and data-gathering. In this setting, the optimizing proposition is satisfied at all locations where data can be uploaded, and the LTL formula specifies a complex data-collection mission. Our method utilizes Büchi automata to produce an automaton (which can be thought of as a graph) whose runs satisfy the temporal-logic specification. We then present a graph algorithm that computes a run corresponding to the optimal robot path. We present an implementation for a robot performing data collection in a road-network platform.This work was supported in part by the Office of Naval Research (grant number MURI N00014-09-1051), Army Research Office (grant number W911NF-09-1-0088), Air Force Office of Scientific Research (grant number YIP FA9550-09-1-020), National Science Foundation (grant number CNS-0834260), Singapore-MIT Alliance for Research and Technology (SMART) Future of Urban Mobility Project and by Natural Sciences and Engineering Research Council of Canada. (MURI N00014-09-1051 - Office of Naval Research; W911NF-09-1-0088 - Army Research Office; YIP FA9550-09-1-020 - Air Force Office of Scientific Research; CNS-0834260 - National Science Foundation; Singapore-MIT Alliance for Research and Technology (SMART); Natural Sciences and Engineering Research Council of Canada

Truncated atomic plane wave method for the subband structure calculations of Moir\'e systems

We propose a highly efficient and accurate numerical scheme named Truncated Atomic Plane Wave (TAPW) method to determine the subband structure of Twisted Bilayer Graphene (TBG) inspired by BM model. Our method utilizes real space information of carbon atoms in the moir\'e unit cell and projects the full tight binding Hamiltonian into a much smaller subspace using atomic plane waves. We present accurate electronic band structures of TBG in a wide range of twist angles together with detailed moir\'e potential and screened Coulomb interaction at the first magic angle using our new method. Furthermore, we generalize our formalism to solve the problem of low frequency moir\'e phonons in TBG

Optimal multi-robot path planning with temporal logic constraints

In this paper we present a method for automatically planning optimal paths for a group of robots that satisfy a common high level mission specification. Each robot's motion in the environment is modeled as a weighted transition system. The mission is given as a Linear Temporal Logic formula. In addition, an optimizing proposition must repeatedly be satisfied. The goal is to minimize the maximum time between satisfying instances of the optimizing proposition. Our method is guaranteed to compute an optimal set of robot paths. We utilize a timed automaton representation in order to capture the relative position of the robots in the environment. We then obtain a bisimulation of this timed automaton as a finite transition system that captures the joint behavior of the robots and apply our earlier algorithm for the single robot case to optimize the group motion. We present a simulation of a persistent monitoring task in a road network environment.United States. Office of Naval Research. Multidisciplinary University Research Initiative (N00014-09-1051)United States. Army Research Office (W911NF-09-1-0088)United States. Air Force Office of Scientific Research (YIP FA9550-09-1-020)National Science Foundation (U.S.). (CNS- 0834260

Realization of SOC behavior in a dc glow discharge plasma

Experimental observations consistent with Self Organized Criticality (SOC) have been obtained in the electrostatic floating potential fluctuations of a dc glow discharge plasma. Power spectrum exhibits a power law which is compatible with the requirement for SOC systems. Also the estimated value of the Hurst exponent (self similarity parameter), H being greater than 0.5, along with an algebraic decay of the autocorrelation function, indicate the presence of temporal long-range correlations, as may be expected from SOC dynamics. This type of observations in our opinion has been reported for the first time in a glow discharge system.Comment: Key Words: Glow discharge; Self Organized Criticality; Hurst exponent; R/S technique; Power spectrum; Autocorrelation function; Nongaussian probability distribution function. Phys Lett A (article in Press

Least Squares Temporal Difference Actor-Critic Methods with Applications to Robot Motion Control

We consider the problem of finding a control policy for a Markov Decision Process (MDP) to maximize the probability of reaching some states while avoiding some other states. This problem is motivated by applications in robotics, where such problems naturally arise when probabilistic models of robot motion are required to satisfy temporal logic task specifications. We transform this problem into a Stochastic Shortest Path (SSP) problem and develop a new approximate dynamic programming algorithm to solve it. This algorithm is of the actor-critic type and uses a least-square temporal difference learning method. It operates on sample paths of the system and optimizes the policy within a pre-specified class parameterized by a parsimonious set of parameters. We show its convergence to a policy corresponding to a stationary point in the parameters' space. Simulation results confirm the effectiveness of the proposed solution.Comment: Technical report accompanying an accepted paper to CDC 201

Multi-UAV Convoy Protection: An Optimal Approach to Path Planning and Coordination

(c) 2010 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.Digital Object Identifier : 10.1109/TRO.2010.2042325In this paper we study the problem of controlling a group of Unmanned Aerial Vehicles (UAVs) to provide convoy protection to a group of ground vehicles. The UAVs are modeled as Dubins vehicles flying at a constant altitude with bounded turning radius. We first present time-optimal paths for providing convoy protection to stationary ground vehicles. Then we propose a control strategy to provide convoy protection to ground vehicles moving on straight lines. The minimum number of UAVs required to provide perpetual convoy protection in both cases are derived

Optimal Motion Primitives for Multi-UAV Convoy Protection

(c) 2010 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.Digital Object Identifier : 10.1109/ROBOT.2010.5509221In this paper we study the problem of controlling a number of Unmanned Aerial Vehicles (UAVs) to provide convoy protection to a group of ground vehicles. The UAVs are modeled as Dubins vehicles flying at a constant altitude with bounded turning radius. This paper first presents time-optimal paths for providing convoy protection to static ground vehicles. Then this paper addresses paths and control strategies to provide convoy protection to ground vehicles moving on a straight line. Minimum numbers of UAVs required to provide perpetual convoy protection for both cases are derived